We propose and investigate new complementary methodologies for estimating predictive variance networks in regression neural networks. We derive a locally aware mini-batching scheme that results in sparse robust gradients, and we show how to make unbiased weight updates to a variance network. Further, we formulate a heuristic for robustly fitting both the mean and variance networks post hoc. Finally, we take inspiration from posterior Gaussian processes and propose a network architecture with similar extrapolation properties to Gaussian processes. The proposed methodologies are complementary, and improve upon baseline methods individually. Experimentally, we investigate the impact of predictive uncertainty on multiple datasets and tasks ranging from regression, active learning and generative modeling. Experiments consistently show significant improvements in predictive uncertainty estimation over state-of-the-art methods across tasks and datasets.

We propose and investigate new complementary methodologies for estimating predictive variance networks in regression neural networks.

We propose and investigate new complementary methodologies for estimating predictive variance networks in regression neural networks.

We thank the reviewers for their constructive and fair reviews. We will discuss two shared concerns, and then move to individual reviewers. This is an interesting venue for further research. We can rely on fast approximate nearest neighbor algorithms. Thus, the two processes can in principle be performed in parallel.

Post-Rebuttal Feedback Thank the reviewers for your feedback. I think this is a good paper to appear in NeurIPS. This paper tackles the uncertainty prediction via directly predicting the marginal mean and variances. For assuring the reliability of its uncertainty estimation, the paper presents a series of interesting techniques for training the prediction network, including location-aware mini-batching, mean-variance split training and variance networks. With all these techniques adopted, the paper demonstrates convincing empirical results on its uncertainty estimation. Weakness, I am surprised by the amazing empirical performance and the simplicity of the method.

The authors identify problems with estimating predictive variance using neural networks, and propose solutions to fix them. All the reviewers agreed that the paper is well-written, clearly highlighting the limitations of current methods and demonstrating that the proposed solution works better. The reviewers gave some suggestions to improve the paper, and raised some questions about computational complexity and scalability to high dimensions. I encourage the authors to take these into account when they prepare the final version.

We propose and investigate new complementary methodologies for estimating predictive variance networks in regression neural networks. We derive a locally aware mini-batching scheme that results in sparse robust gradients, and we show how to make unbiased weight updates to a variance network. Further, we formulate a heuristic for robustly fitting both the mean and variance networks post hoc. Finally, we take inspiration from posterior Gaussian processes and propose a network architecture with similar extrapolation properties to Gaussian processes. The proposed methodologies are complementary, and improve upon baseline methods individually. Experimentally, we investigate the impact of predictive uncertainty on multiple datasets and tasks ranging from regression, active learning and generative modeling.

In model-free deep reinforcement learning (RL) algorithms, using noisy value estimates to supervise policy evaluation and optimization is detrimental to the sample efficiency. As this noise is heteroscedastic, its effects can be mitigated using uncertainty-based weights in the optimization process. Previous methods rely on sampled ensembles, which do not capture all aspects of uncertainty. We provide a systematic analysis of the sources of uncertainty in the noisy supervision that occurs in RL, and introduce inverse-variance RL, a Bayesian framework which combines probabilistic ensembles and Batch Inverse Variance weighting. We propose a method whereby two complementary uncertainty estimation methods account for both the Q-value and the environment stochasticity to better mitigate the negative impacts of noisy supervision. Our results show significant improvement in terms of sample efficiency on discrete and continuous control tasks.

Deep neural networks tend to underestimate uncertainty and produce overly confident predictions. Recently proposed solutions, such as MC Dropout and SDENet, require complex training and/or auxiliary out-of-distribution data. We propose a simple solution by extending the time-tested iterative reweighted least square (IRLS) in generalised linear regression. We use two sub-networks to parametrise the prediction and uncertainty estimation, enabling easy handling of complex inputs and nonlinear response. The two sub-networks have shared representations and are trained via two complementary loss functions for the prediction and the uncertainty estimates, with interleaving steps as in a cooperative game. Compared with more complex models such as MC-Dropout or SDE-Net, our proposed network is simpler to implement and more robust (insensitive to varying aleatoric and epistemic uncertainty).